System and method for device identification and uniqueness

ABSTRACT

Systems and methods for determining uniqueness of device identifiers are provided are provided. The uniqueness of a device identifier may be indicated by a device quality score or grade that is calculated based on a plurality of parameters associated with a device identifier as well as evaluation rules derived based on historical data. The plurality of parameters may be associated with a network event or transaction associated with the device identifier. The evaluation rules may be derived using machine learning techniques. Based on uniqueness of a device identifier, a suitable action or measure may be taken.

CROSS-REFERENCE

This application claims the benefit of U.S. Provisional Application No.61/872,287, filed Aug. 30, 2013, which application is incorporatedherein by reference.

BACKGROUND OF THE INVENTION

With rapid advancement of computer technologies and e-Commerce, peopleare increasingly reliant on a variety of internet-connected devices foreverything from banking to booking travel to shopping. As a serviceprovider, it has become increasingly important to distinguish among thedifferent devices in order to provide detect and prevent online fraudand/or to provide customized content or services.

SUMMARY OF THE INVENTION

System and methods for determining uniqueness of device identifiers areprovided. According to an aspect of the invention, acomputer-implemented method for determining uniqueness of a deviceidentifier is provided. The method comprises obtaining a plurality ofevaluation rules based at least in part on historical data anddetermining the uniqueness of the device identifier based at least inpart on the evaluation rules and a plurality of parameters associatedwith the device identifier. Obtaining the one or more evaluation rulesmay include analyzing the historical data using a machine learningtechnique. Determining the uniqueness of the device identifier mayinclude determining a device quality score associated with the deviceidentifier. Determining the uniqueness of the device identifier mayinclude selecting a subset of the one or more evaluation rules based atleast in part on the plurality of parameters and applying the subset ofevaluation rules to at least some of the plurality of parameters toobtain the device quality score. The method may further comprisedetermining a suitable action based at least in part on the uniquenessof the device identifier. Determining the suitable action may includeselecting a first action if the device identifier is more likely to beunique and selecting a second action if the device identifier is lesslikely to be unique.

INCORPORATION BY REFERENCE

All publications, patents, and patent applications mentioned in thisspecification are herein incorporated by reference to the same extent asif each individual publication, patent, or patent application wasspecifically and individually indicated to be incorporated by reference.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth with particularity inthe appended claims. A better understanding of the features andadvantages of the present invention will be obtained by reference to thefollowing detailed description that sets forth illustrative embodiments,in which the principles of the invention are utilized, and theaccompanying drawings of which:

FIG. 1 illustrates an example environment for implementing the presentinvention, in accordance with an embodiment.

FIG. 2a illustrates example components of a device identificationsystem, in accordance with an embodiment.

FIG. 2b illustrates example components of a device identificationsystem, in accordance with another embodiment.

FIG. 3 illustrates example components of a computer device forimplementing aspects of the present invention, in accordance with anembodiment.

FIG. 4 illustrates an example process for implementing the presentinvention, in accordance with an embodiment.

FIG. 5 illustrates an example process for determining the uniqueness ofa device identifier, in accordance with an embodiment.

FIG. 6 illustrates an example process for determining the uniqueness ofa device identifier, in accordance with an embodiment.

FIG. 7 illustrates an example process for calculating a device qualityscore, in accordance with an embodiment.

FIG. 8 illustrates an example process for determining the uniqueness ofa device identifier, in accordance with an embodiment.

DETAILED DESCRIPTION OF THE INVENTION

According to aspects of the present invention, a device identificationsystem and methods may be provided for determining the uniqueness ofdevice identifiers. Network devices, such as desktops, laptops, tabletcomputing devices, smart phones, smart TVs, and the like, may beidentified using device identifiers. Such device identifiers may beprovided or generated by the manufacturers, distributors, developers, orany suitable entity. Examples of device identifier may include Androididentifier (ID), iPhone's Unique Identifier (UDID), iPhone'sIdentifierForAdvertising (IFA or IDFA), cookie ID, login ID, InternetProtocol (IP) address, media access control (MAC) address, a hash of anyof the above, a combination of any of the above, or the like. In somecases, the device identifier may be derived based on one or morehardware and/or software parameters of a device identified by the deviceidentifier. For example, a device identifier may be derived from the IPaddress, operating system version, and locale setting of the device. Insome embodiments, a device identifier may be used to identify the sourceor origin or a transaction, request, or network event. For example, adevice identifier may include a user identifier, an account identifier,and the like. Ideally, a device identifier uniquely identifies a device.In other words, different devices are mapped to different deviceidentifiers, for example, based on unique software/hardwarecharacteristics associated with the devices. However, in some cases,different devices may have the same device identifiers. In some cases,such as in an online fraud, such sharing of device identifiers may beintentional. In some other cases, such sharing of device identifiers maybe unintentional.

In some embodiments, a device identifier may be used to distinguishamong the entities (e.g., users) associated with the devices identifiedby the device identifier. For example, a content or service provider mayuse such device identifiers to distinguish among different users so asto provide customized advertisement or service items tailored to thepreferences of the users. However, such targeted user-specific action(e.g., target advertisement delivery) is only effective when there is ahigh probability that the device identifier is indeed unique, that is,it is very likely that the device identifier is not shared by multipledevices. In some instances, different devices may have the same deviceidentifiers. For example, the Android ID for two Android devices may bethe same. For another example, two devices may have the same IP address.As yet another example, devices having similar or different parametersmay result in having the same value for their device identifiers. Insuch cases when the device identifier is not unique, targeted action maynot be feasible or desirable. Rather, a different action or approach maybe required.

As an example, consider two users Carola and Marley. Carola operates adevice 1 to access a shopping website and Marley operates a device 2 toaccess the same shopping site. Carola is interested in jewelry and artand Marley is interested in cars and technology. The service provideroperating the shopping website may be able to deliver different targetedadvertisement to Carola and Marley based on their different preferencesif the service provider can distinguish the device 1 from device 2. Inother words, given a device identifier, if the service provider can beconfident that the device identifier is unique, that is, it is unique toa specific device, then the service provider's may be able to provideeffective targeted content. On the other hand, if for a given the deviceidentifier, the service provider is not confident that the deviceidentifier is unique (i.e., associated with only with Carola or onlywith Marley), then it may be undesirable to delivery customized contentbecause it may offend or otherwise alienate the unintended audience ofsuch customized content. Rather, providing more generic content mayprove to be more effective in this case.

As another example, a fraud detection engine may be configured to detectfraudulent activities associated of network devices. For a given deviceidentifier associated with fraudulent transactions, if it is likely thatthe device identifier is unique, it may be desirable to take specificfraud prevention actions with respect to activities associated with thedevice identifier. For example, the device identifier may be added to ablacklist so that activities associated with the device identifier areblocked. This may be a desirable solution that prevents future fraudwithout affecting non-rogue devices since it is unlikely that the deviceidentifier is shared by other devices. Conversely, if the deviceidentifier is likely to be non-unique, then put the device identifierinto a blacklist may unduly impact legitimate activities of non-roguedevices that happen to share the same device identifier. Rather, a moremoderate measure may be taken such as monitoring instead of blockingtraffic coming from devices identified by the device identifier.

According to aspects of the present invention, a device identificationsystem and methods may be provided for determining the uniqueness ofdevice identifiers. The device identification system may include ananalysis engine, an evaluation engine, and optionally, an action engine(such as illustrated in FIG. 2a below). The analysis engine may beconfigured to generate, based on historical data, rules that may be usedto determine the uniqueness of device identifiers. Specifically, therules may be used to derive a quality score associated with a givendevice identifier. Quality score may be a numerical value. Forinstances, the quality score may be any integer between 0 and 100(inclusive) where the higher the quality score the more likely that thedevice identifier is unique. The rules may be stored in a data storewhich may include a database or a data file (e.g., a JSON file) that isaccessible to the evaluation engine.

The evaluation engine may be configured to receive a plurality ofparameters associated with a device identifier, select the applicablerules (e.g., based on at least some of the parameters such as thedevice's operating system or hardware type) and apply the rules to theplurality of parameters to derive a quality score and/or a qualitygrade. In some cases, the quality grade may be derived based on thequality score. In some embodiments, the plurality of parameters mayinclude any hardware and/or software parameters associated with a deviceidentified by the device identified.

In some embodiments, the plurality of parameters may includedeterministic identifiers such as customer ID, login ID, account ID,cookie ID, UDID, Android ID, IFA or IFDA, Identifier For Vendor (IFV),International Mobile Equipment Identity (IMEI), MAC address, IP address,and the like.

In some embodiments, the plurality of parameters may include clientand/or server location and/or geographical information; client devicetrustability score or similar indicator; user lifetime value (LTV) orsimilar indicator; client device Return on Investment Index (ROlndex) orsimilar indicator; device metadata such as manufacturer (e.g., Apple,Samsung, Microsoft, Dell, etc), name (e.g., iPhone), model, version, andthe like; device operating system (e.g., iOS, Android, Windows Phone,BlackBerry, Mac OS, OS X, Microsoft Windows, Unix, Linux, BSD, etc.);browser metadata such as maker (e.g., Google, Microsoft, Mozilla), name(e.g., Chrome, Internet Explorer, Firefox, Opera, Safari), version, andthe like; other software and/or hardware characteristics (e.g., AdobeFlash version); event type such as impression, click or selection,download, page load, and the like; event type detail such as campaigntype, transaction type, and the like; customer type and industry; andother parameters.

In typical embodiments, such parameters are obtained without theawareness of the user operating the device. In some cases, some or allof the parameters may be obtained with the user's awareness. In someembodiments, some or all of the parameters may be included in one ormore requests or messages provided by the device or by another entitysuch as a Domain Name System (DNS) server, an Active Directory (AD)server, and the like. Example parameters may include user agent (UA), IPaddress, user identity, user credentials, network protocol, serviceendpoint, Hypertext Transfer Protocol (HTTP) method and/or status code,operating system, locale or language code, processor architecture,device type (e.g., desktop, mobile phone, etc.) and the like. In someembodiments, some or all of the parameters discussed herein may be usedto derive the device identifier and to derive the quality score orgrade. For example, the parameters may be concatenated, combined,appended, hashed, encrypted, and otherwise processed to derive thedevice identifier and/or quality score or grade. For example, in anembodiment, the device identifier may include a 40-character SHA-1 hashof some of the parameters.

In some embodiments, a set of applicable rules may be selected based atleast in part on some of the parameters associated with the deviceidentifier. For example, different sets of rules or the same set ofrules may be provided for different types of devices and/or operatingsystems. For example, a first set of rules may be selected for anAndroid device whereas a second set of rules may be selected for an iOSdevice.

In an embodiment, the rules include, for each of at least some of theplurality of parameters, a corresponding value-weight map. In somecases, the rules may include usage rules associated with thevalue-weight maps that specify how the maps should be applied to theparameter. In other embodiments, such usage rules may be optional. Eachof the value-weight maps may include one or more parameter values orvalue ranges along with corresponding weights. The parameter values mayinclude any of the above-discussed parameters or a derivation thereof.The table below provides an example value-weight map for the user agentparameter:

Parameter Value Weight 8 0.5 10 0.38 12 0.01 . . . . . .

For the above example, a usage rule associated with the value-weight mapmay specify that the parameter value to be used to look up thevalue-weight map is the length of the user agent identifier characterstring. For example, given a user agent parameter of “UA_2.8.1”, thecorresponding parameter value, according to the “UA-length” rule, is 8,the length of the character string “UA_2.8.1” and the correspondingweight is 0.5.

Thus, for at least some of the plurality of parameters associated withthe device identifier, corresponding weights may be obtained by applyingthe rules (e.g., by looking up the corresponding parameter-specificvalue-weight map as specified by usage rule).

While the value-weight maps illustrated here each corresponds to aspecific parameter, in some embodiments, a value-weight map maycorrespond to more than one parameter. For example, the value used tolook up the value-weight map may be derived based on the values of oneor more parameters.

In some embodiments, the weights associated with the parameters may beweighted, for example, based on the perceived importance of theparameters. The perceived importance of the parameters may be determinedbased on statistical analysis of the historical data. For example, in anembodiment, the weight associated with the IP address parameter may begiven a larger weight than the weight associated with the user agentparameter.

A quality score indicative of the uniqueness of a device identifier maybe derived based on the parameter weights, which may be weighted asdiscussed above. For example, the device score may be calculated as alinear combination of the weighted weight values. The quality score maybe categorized into quality grades. For example, a quality score between80 and 100 may be categorized as quality grade A, a quality scorebetween 60 and 80 may be categorized as quality grade B, and so on.Thus, device identifiers may be segmented according to their qualitygrades. In general, a device identifier is considered to be “highquality” if it has a high quality score or grade and “low quality” if ithas a low quality score or grade.

In some embodiments, the value-weight maps, usage rules, formula and/oralgorithm for calculating the quality scores and the like arecollectively referred to as the evaluation rules (or rules). Some or allof such evaluation rules may be derived based on the historical dataassociated with past user activities and usage of computing resources.In particular, the historical data may be analyzed using statisticalanalysis and machine learning techniques such as logistical regression.Other suitable data mining techniques may also be used. Such dataanalysis may be performed by the analysis engine with or without humanintervention.

As discussed above, the calculation of the quality scores or grades doesnot involve analyzing vast amount of historical data. Rather, thecalculation is performed based on the rules derived from the historicaldata. The size of the rules may be significantly smaller than the sizeof the historical data which the rules are based on. For example, therules may fit in one or more JSON files whereas the historical data maybe stored in large data storage systems. Given the pre-calculated orderived rules, the time and complexity of the quality score/gradecalculation (and hence determination of uniqueness of deviceidentifiers) is significantly reduced. In some cases, uniquenessdetermination may be performed efficiently for a large amount oftransactions in a short period of time. As the historical data evolveover time, the rules derived from the historical data may be updated toreflect any changes (e.g., on a periodic basis). By using such updatedrules, the quality score/grade calculation also reflects the changes inhistorical data.

In some embodiments, the device identification system discussed hereinmay or may not include an action engine. The action engine may beconfigured to take different actions based on the uniqueness of deviceidentifiers (such as indicated by the quality grades or quality scores).For example, the action engine may be configured to provide moretargeted content to devices associated with a high quality deviceidentifier and less targeted content to devices associated with a lowerquality device identifier. It shall be understood that different aspectsof the invention can be appreciated individually, collectively, or incombination with each other.

FIG. 1 illustrates an example environment 100 for implementing thepresent invention, in accordance with an embodiment. As illustrated, oneor more user devices 102 connect via a network 104 to a deviceidentification system 106 configured to provide device identificationfunctionalities described herein. In various embodiments, the userdevices 102 may include any devices capable of communicating with thenetwork 104, such as personal computers, workstations, laptops,smartphones, mobile phones, tablet computing devices, smart TVs, gameconsoles, internet-connected setup boxes, kitchen appliances and thelike. In some embodiments, the user devices 102 may include applicationssuch as web browsers and/or applications (e.g., mobile apps) that arecapable of communicating with the device identification system 106and/or a system that uses the device identification system 106.

In some embodiments, the device identification system 106 may include orbe included in one or more computing systems. For example, the deviceidentification system 106 may be a part of a content provider. Forexample, the device identification system 106 may be a runtime componentof a web server of the content server. As another example, the deviceidentification system 106 may be a part of a fraud detection system orservice used by an online service provider such as a bank, a merchant, apayment service provider, and the like. In some embodiments, the deviceidentification system may be owned and/or operated by the same ordifferent entity as the content provider.

In some embodiments, the device identification system 106 may beimplemented by one or more physical and/or logical computing devices orcomputer systems that collectively provide the functionalities describedherein. For example, aspects of the device identification system 106 maybe implemented by a single server or by a plurality of servers (e.g.,distributed Hadoop nodes). As another example, aspects of the deviceidentification system 106 may be implemented by one or more processesrunning on one or more devices. In some embodiments, the deviceidentification system 106 may provide an API such as a web serviceinterface that may be used by users or other processes or services toutilize the functionalities of the device identification systemdiscussed herein.

In some embodiments, the device identification system 106 may compriseone or more computing services provisioned from a “cloud computing”provider, for example, Amazon Elastic Compute Cloud (“Amazon EC2”),provided by Amazon.com, Inc. of Seattle, Wash.; Sun Cloud ComputeUtility, provided by Sun Microsystems, Inc. of Santa Clara, Calif.;Windows Azure, provided by Microsoft Corporation of Redmond, Wash., andthe like.

In some embodiments, the device identification system 106 maycommunicate with a data store 108 in order to perform thefunctionalities described herein. For example, the data store 108 may beused to store historical data, evaluation rules, and the like.

In some embodiments, the data store 108, or any other data storesdiscussed herein, may include one or more data files, databases (e.g.,SQL database), data storage devices (e.g., tape, hard disk, solid-statedrive), data storage servers, or the like. In various embodiments, sucha data store 108 may be connected to the device identification system106 locally or remotely via a network. In some embodiments, data store108, or any other data stores discussed herein, may comprise one or morestorage services provisioned from a “cloud storage” provider, forexample, Amazon Simple Storage Service (“Amazon S3”), provided byAmazon.com, Inc. of Seattle, Wash., Google Cloud Storage, provided byGoogle, Inc. of Mountain View, Calif., and the like.

In various embodiments, the network 104 may include the Internet, alocal area network (“LAN”), a wide area network (“WAN”), a cellularnetwork, wireless network or any other public or private data and/ortelecommunication network.

FIG. 2a illustrates example components of a device identification system200A, in accordance with an embodiment. The device identification system200A may be similar to the device identification system 106 discussed inFIG. 1. In various embodiments, the device identification system 200Amay include one or more components that individually or collectivelyprovide a set of functionalities. Each component may be implemented byone or more physical and/or logical computing devices, such ascomputers, data storage devices and the like. Some or all of thecomponents may be co-located on the same device or distributed ondifferent devices. The components may communicate with each other orwith external entities such as other systems, devices or users. It willbe appreciated by those of ordinary skill in the art that variousembodiments may have fewer or a greater number of components orsubcomponents than those illustrated in FIG. 2a . Thus, the depiction ofthe environment in FIG. 2a or in other figures should be taken as beingillustrative in nature and not limiting to the scope of the disclosure.

In the illustrated embodiment, the device identification system 200Aincludes an analysis engine 202, evaluation engine 204 and an actionengine 206. In some other embodiments, the device identification system200A may include a subset or a superset of the illustrated components.For example, in an embodiment, the device identification system mayinclude only the evaluation engine. In another embodiment, the deviceidentification system may include only the analysis engine and theevaluation engine. In some embodiments, some or all of the componentsdiscussed herein may be combined or further divided into subcomponents.Some or all of the components may be implemented by the provider of thesystem or by a third party service provider.

The analysis engine 202 may be configured to generate, based onhistorical data 201, evaluation rules or rules 208 that may be used todetermine the uniqueness of device identifiers. Specifically, the rulesmay be used, for example, by the evaluation engine 204, to derive aquality score or grade 210 associated with a given device identifier.Such rules may be derived based on historical data obtained from manyuser devices and many transactions. Various techniques may be used toderive the rules including machine learning techniques, neural networks,fuzzy logic, statistical analysis (e.g., logistical regression), and thelike. Rules may be generated automatically with aid of a processor.Human intervention may or may not be required for generating the rules.

The historical data may include data (including statistics) related topast user activities, transactions, requests, responses, usage ofcomputing resources and the like. In some cases, the historical data mayinclude information indicative of reliability, trustworthiness oruniqueness of user devices. For example, the historical data mayindicate that a certain IP address or a certain operating system issusceptible to security problems (e.g., virus, Denial of Service (DoS)attack, session hijacking, Man-in-the-Middle (MITM) orMan-in-the-Browser (MITB) attacks, etc.). As another example, thehistorical data may indicate that certain types of devices tend to sharethe same device identifiers.

The evaluation engine 204 may be configured to determine uniqueness of adevice identifier based on evaluation rules 208, discussed above. Tothat end, the evaluation engine 204 may be configured to obtain devicedata 205 associated with a device. In some embodiments, the device datamay include a plurality of parameters associated with or used to derivea device identifier. In some cases, device data may include the deviceidentifier itself. The plurality of parameters may include any hardwareand/or software parameters associated with a device identified by thedevice identified such user agent identifier, IP address, user identityinformation, user credentials, network protocols, service endpoint,service method, HTTP method and/or status code, operating system, localeor language code, processor architecture, device type (e.g., desktop,mobile phone, etc.) and the like. In some embodiments, the plurality ofparameters may be associated with a particular transaction or networkevent.

Based at least in part on the plurality of parameters (e.g., devicetype), the evaluation engine 204 may be configured to select and applysome or all of the evaluation rules 208 made available by the analysisengine 202. In some embodiments, the evaluation rules may be stored in adata store or data file that is made available to the evaluation engine204. The evaluation rules may be applied to at least some of theparameters to derive a device quality score or grade using methodsdiscussed herein.

In some embodiments, the rules may be used to determine quality scoreand/or quality grade without requiring access to historical data. Suchdetermination may be performed, for example, by the evaluation engine.Such a rules-only approach may be beneficial. For example, in somecases, the historical data may include sensitive or personallyidentifying information such as credit card information. In such cases,it may be undesirable to allow certain entities to have access to thehistorical data, for example, for privacy concerns.

The device quality score or grade may be used by the action engine 206to determine an action 207. In various embodiments, the action enginemay include or be included in one or more web servers, data servers,security and/or fraud detection servers and the like. The action mayinclude retrieval, storage, processing, modification, transmission, orthe like, of one or more responses to a request, internal or externalmessages or instructions, content data, and the like. In some cases,device identification system discussed herein may be used to detectfraudulent and/or malicious attacks such as session hijacking, MITM/MITBattacks, harvesting P2P networks, and the like. In some cases, deviceidentification system may be used to determine suitable content (e.g.,advertisement) to provide.

In some embodiments, analysis engine, the evaluation engine and theaction engine may reside on the same or different computing devices andmay each be implemented by one or more computing devices or processes.In some embodiments, the rules, the device quality scores or grades,and/or the actions may be generated in real or nearly real time as thedata is coming in, or in an asynchronous fashion such as in using batchprocessing. In some embodiments, the generation of rules and theevaluation of the uniqueness of device identifiers can be independentfrom each other. The rules may be generated and/or updated at adifferent time schedule than that for the evaluation of the deviceidentifiers. For example, in an embodiment, the rules are generatedahead of time and updated on a periodic basis. Independently orasynchronously to the generation and/or update of rules, deviceidentifiers may be evaluated in real or nearly real time using therules.

In some embodiments, analysis engine, the evaluation engine and theaction engine may be configured to provide the various functionalitiesdiscussed herein in a synchronous or asynchronous fashion. For example,the generation of rules may be performed offline, in an asynchronousfashion. The evaluation of device quality score or grade may beperformed in real time or nearly real time as the device data isreceived. The determining of a suitable action based on the device scoreand/or grade may be performed in real time or nearly real time.

FIG. 2b illustrates example components of a device identification system200B, in accordance with another embodiment. In this example, the deviceidentification system 200B includes a data collector 214 residing on auser or client device 212. The data collector may be implemented as abrowser script using JavaScript or any other scripting language. Thedata collector may be configured to communicate with a deviceidentification service 216. For example, the data collector may beconfigured to collect parameter information about the user device suchas discussed herein and transmit such parameter information to thedevice identification service 216, for example, using an API provided bythe device identification service. In some embodiments, the collectionand/or communication with the device identification service may betriggered by an event such as a browser event. For example, the eventmay include a click on a portion (e.g., a button or a link) of a webpage, loading of a web page and the like.

The device identification system 200B includes a device identificationservice 216 that may be implemented as a web service. The deviceidentification service 216 may be implemented by one or more servers.The servers implementing the device identification service 216 may beowned and/or provided by a content or service provider for the userdevice (e.g., banking, ecommerce, retail) or by the provider of thedevice identification system 200B.

In some embodiments, the device identification service 216 may beconfigured to receive parameter information provided by the datacollector of the user device and to provide a device identifier and/ordevice quality score or grade based on the parameter information. Tothat end, the device identification service 216 may utilize anevaluation engine 218. The evaluation engine 218 may be configured tocalculate a device identifier and/or a device quality score or gradebased on the parameter information. In some embodiments, the evaluationengine 218 may be implemented using one or more server-side libraryfiles.

In some embodiments, some or all of the parameters may be used to derivethe device identifier. For example, the parameters may be concatenated,combined, appended, hashed, encrypted, and otherwise processed. Forexample, in an embodiment, the device identifier may include a40-character SHA-1 hash.

In some embodiments, the device quality score or grade may be evaluatedbased on some or all of the parameters. For example, a lookup table(e.g., stored in memory) may be used to determine the weight valuesassociated with some or all of the parameters. The weight values may ormay not be further weighted, combined or otherwise processed to derive afinal device quality score or grade. The device quality score may becategorized into a device quality grade. In some embodiments, the lookuptable and the algorithm for deriving the quality score or grade may beincluded on one or more rules that are pre-determined based onhistorical data such as past transactions and/or user activities relatedto one or more websites or web services. Thus, access to the actualhistorical data may not be required for the evaluation of the qualityscores or grades. In some embodiments, the generation of the deviceidentifiers and/or the associated device quality scores and/or gradesmay be performed in real time or nearly real time with respect to thereceipt of the parameter information. In other embodiments, any or allof the above operations may be performed in an asynchronous mode, forexample, using batch processing.

In some embodiments, the generated device identifier and associateddevice quality score and/or grade may be stored in a data store 220. Thedata store 220 may include a user ID map (not shown) or a similar datastructure configured to store a mapping between device identifiers anddevice quality scores and/or grades. In some embodiments, the data store220 may include a memory of a server, one or more data storage device(e.g., SSD, hard disk, taps), or a cloud-based storage service such asdiscussed in connection with FIG. 1. The data store 220 may or may notbe owned and/or operated by the same as the provider of the deviceidentification service 216. For example, the user ID map may be storedat least in part on a customer server and/or a fraud-detection system.

In some embodiments, the storing of the device identifiers and/or theassociated device quality scores and/or grades may be performed in realtime or nearly real time as the above information is generated. In otherembodiments, any or all of the above operations may be performed in anasynchronous mode, for example, using batch processing.

In various embodiments, the user ID map may be used by any suitableentity for any suitable purpose. For example, in an embodiment, the userID map may be used by a content provider to determine the type ofcontent to provide to a user device. More targeted content (e.g.,advertisement) may be provided for device identifiers with higherquality grades and less targeted content may be provided for deviceidentifiers with lower quality grades. In another embodiment, the userID map may be used by a fraud detection system to detect and/or preventonline fraud.

In some embodiments, the user ID map may be used to update and/or refinethe evaluation rules (e.g., including weight lookup table, device scorecomputation algorithm) discussed herein. For example, the user ID mapmay be provided for research purposes. The research may be performed bya provider of the device identification system or a third party serviceprovider.

FIG. 3 illustrates example components of a computer device 300 forimplementing aspects of the present invention, in accordance with anembodiment. In another embodiment, the computer device 300 may beconfigured to implement a user device such as a user device 102discussed in connection with FIG. 1 and/or components or aspects of thedevice identification system such as described in connection with FIGS.1 and 2. In some embodiments, computing device 300 may include many morecomponents than those shown in FIG. 3. However, it is not necessary thatall of these components be shown in order to disclose an illustrativeembodiment.

As shown in FIG. 3, computing device 300 includes a network interface302 for connecting to a network such as discussed above. In variousembodiments, the computing device 300 may include one or more networkinterfaces 302 for communicating with one or more types of networks suchas the Internet, wireless networks, cellular networks, and any othernetwork.

In an embodiment, computing device 300 also includes one or moreprocessing units 304, a memory 306, and an optional display 308, allinterconnected along with the network interface 302 via a bus 310. Theprocessing unit(s) 304 may be capable of executing one or more methodsor routines stored in the memory 306. The display 308 may be configuredto provide a graphical user interface to a user operating the computingdevice 300 for receiving user input, displaying output, and/or executingapplications. In some cases, such as when the computing device 300 is aserver, the display 308 may be optional.

The memory 306 may generally comprise a random access memory (“RAM”), aread only memory (“ROM”), and/or a permanent mass storage device, suchas a disk drive. The memory 306 may store program code for an operatingsystem 312, one or more device identification routines 314, and otherroutines. In various embodiments, the program code may be stored on acomputer-readable storage medium, for example, in the form of a computerprogram comprising a plurality of instructions executable by one or moreprocessors. The computer-readable storage medium may be non-transitory.The one or more device identification routines 314, when executed, mayprovide various functionalities associated with the deviceidentification system as described herein.

In some embodiments, the software components discussed above may beloaded into memory 306 using a drive mechanism associated with anon-transient computer readable storage medium 318, such as a floppydisc, tape, DVD/CD-ROM drive, memory card, USB flash drive, solid statedrive (SSD) or the like. In other embodiments, the software componentsmay alternatively be loaded via the network interface 302, rather thanvia a non-transient computer readable storage medium 318. In anembodiment, the computing device 300 also include an optional timekeeping device (not shown) for keeping track of the timing oftransactions or network events.

In some embodiments, the computing device 300 also communicates via bus310 with one or more local or remote databases or data stores such as anonline data storage system via the bus 310 or the network interface 302.The bus 310 may comprise a storage area network (“SAN”), a high-speedserial bus, and/or via other suitable communication technology. In someembodiments, such databases or data stores may be integrated as part ofthe computing device 300.

FIG. 4 illustrates an example process 400 for implementing the presentinvention, in accordance with an embodiment. Aspects of the process 400may be performed, for example, by a device identification system such asdiscussed in connection with FIGS. 1 and 2 or one or more computingdevices such as discussed in connection with FIG. 3. Some or all aspectsof the process 400 (or any other processes described herein, orvariations and/or combinations thereof) may be performed under thecontrol of one or more computer/control systems configured withexecutable instructions and may be implemented as code (e.g., executableinstructions, one or more computer programs or one or more applications)executing collectively on one or more processors, by hardware orcombinations thereof. The code may be stored on a computer-readablestorage medium, for example, in the form of a computer programcomprising a plurality of instructions executable by one or moreprocessors. The computer-readable storage medium may be non-transitory.The order in which the operations are described is not intended to beconstrued as a limitation, and any number of the described operationsmay be combined in any order and/or in parallel to implement theprocesses.

In an embodiment, the process 400 includes obtaining 402 a set of rulesbased on historical data. In various embodiments, the rules may includethe evaluation rules, discussed herein, that may be used to determinethe uniqueness of a device identifier. For example, the rules mayinclude one or more parameter maps that map parameter values (originalor derived) to weight values. The rules may further include formulas,algorithms, and the like for using the maps to (e.g., combining theweight values) to derive a device quality score and/or device qualitygrade. Such a device quality score or grade may be indicative of theuniqueness of the device identifier. In a typical embodiment, the sizeof the rules is a fraction of the amount of the historical data based onwhich the rules are derived.

As discussed above, historical data may include any data related to pastdata transactions, user activities, usage of computing and networkresources and the like. In some cases, the historical data may includeinformation indicative of reliability, trustworthiness or uniqueness ofuser devices or device identifiers. In some embodiments, the historicaldata may be obtained from a third-party data or service provider and/oraccumulated by a provider of the device identification system. Forexample, historical data may include interactions with contentproviders, ecommerce or online retail service providers, banking, creditcard, or financial service providers, airlines, travel serviceproviders, and the like.

In some embodiments, the rules may be generated using a variety ofmachine learning and/or data mining techniques such as statisticalanalysis, neural networks, and the like. In one embodiment, some of therules may be defined or specified by humans. In some embodiments, someof the rules may be generated from scratch or provided by a third-partyprovider.

In an embodiment, the process 400 includes determining 404 theuniqueness of a device identifier based on the rules discussed above anda plurality of parameters associated with the device identifier. In someembodiments, a device identifier may be used to identify the source ororigin or a transaction, request, or network event. In some embodiments,a device identifier may be determined and/or derived based on any one orcombination of one or more parameters such as described herein. Forexample, the device identifier may be based on one or more hardwareand/or software settings or attributes of a device. For example, adevice identifier may include or be based on an IP address associatedwith an HTTP request. As another example, a device identifier mayinclude or be based on a username associated with an online account anda user agent identifier. The device identifier may include a devicefingerprint without regard to user information. In one embodiment, thedevice identifier may be independent from the parameters describedherein. In a typical embodiment, a device identifier is obtained orderived without the awareness of the originator of the transaction ornetwork event identified by the device identifier. In other embodiments,the device identifier may be obtained with user awareness.

In various embodiments, a plurality of parameters such as thosediscussed herein may be obtained in connection with the deviceidentifier. Such parameters may be obtained from the transaction ornetwork event identified by the device identifier. Such parameters maybe obtained, for example, by analyzing the metadata and/or dataassociated with a request, parsing a network log file, utilizing anysuitable web analytics tools, and the like. In a typical embodiment,such parameters are obtained without user awareness. In someembodiments, such parameters may be collected with user awareness. Insome embodiments, the device identifier and/or parameters may beobtained without downloading anything to the device (i.e., using atag-free technique) or by downloading something (e.g., a cookie orbrowser script) to the device.

Based on the rules and the plurality of parameters, uniqueness of thedevice identifier may be determined. In some embodiments, the uniquenessof the device identifier may be represented by a device quality score orgrade discussed herein. In other embodiments, the uniqueness of thedevice identifier may be represented by any other suitablerepresentations. More details for determining the uniqueness of a deviceidentifier are discussed below in connection with FIGS. 5-7.

In an embodiment, the process 400 includes determining 406 a suitableaction or measure to take based on the uniqueness of the deviceidentifier. Such action may be selected among a plurality of actionsbased on a determined device quality score or quality grade for thedevice identifier. For example, different action(s) may be taken if thedevice identifier is more unique than if the device identifier is lessunique, or if the quality score or grade is different. The action mayinclude an active action such as the retrieval, storage, processing,modification, transmission, or the like, of one or more responses,messages, instructions, and the like. In an embodiment, the action mayinclude not doing something. In some embodiments, determining thesuitable action may include comparing the device quality score or gradewith a predefined threshold value and selecting the suitable actionbased on the result of the comparison. For example, if a deviceidentifier is determined to be more likely to be unique (e.g., having aquality score or grade higher than a predefined threshold value), then amore targeted advertisement may be provided. Conversely, if a deviceidentifier is determined to be less likely to be unique (e.g., having aquality score or grade equal or less than the predefined thresholdvalue), then a less targeted advertisement may be provided. Similarly, amore severe or drastic security or anti-fraud measure (e.g., adding thedevice identifier to a blacklist) may be taken if a device identifier isdetermined to be more likely to be unique. On the other hand, a moremoderate security measure may be taken if a device identifier isdetermined to be less likely to be unique.

In some embodiments, step 402 may be performed on a periodic basis(e.g., daily, weekly, monthly). In some embodiments, steps 404 and 406may be performed for each of a plurality of transactions in real ornearly real time or in an asynchronous fashion (i.e., not in real ornearly real time).

FIG. 5 illustrates an example process 500 for determining the uniquenessof a device identifier, in accordance with an embodiment. Aspects of theprocess 500 may be performed, for example, by the evaluation enginediscussed in connection with FIG. 2a or 2 b.

In an embodiment, the process 500 includes obtaining 502 a set ofevaluation rules based on historical data. As discussed above, suchrules may be made available via a data file, data storage system, webservice, or any other suitable interface. In various embodiments, therules may be made available via the push or pull technologies or acombination of both. In some embodiments, once the set of rules areobtained, they can be used to evaluate the uniqueness of one or more(e.g., hundreds or thousands of) device identifiers. In someembodiments, the rules may be updated occasionally (e.g., on a periodicbasis).

In an embodiment, the process 500 includes obtaining 404 a plurality ofparameters associated with a device identifier. The device identifiermay be associated with a device, a group of devices, a transaction, auser, an organization or any other entity. The plurality of parametersmay be obtained from the device identifier itself, from the entityassociated with the device identifier, from log files, from real-timeanalysis of network traffic, or from other channels using any suitablemethod.

Based on the rules and the plurality of parameters, the uniqueness ofthe device identifier may be determined 506, for example, using theprocess discussed below in connection with FIGS. 6-7.

FIG. 6 illustrates an example process 600 for determining the uniquenessof a device identifier, in accordance with an embodiment. Aspects of theprocess 600 may be performed, for example, by the evaluation enginediscussed in connection with FIG. 2a or 2 b.

In an embodiment, the process 600 includes obtaining 602 a plurality ofparameters associated with a device identifier. In some embodiments,step 602 may be similar to step 504 discussed in connection with process500 of FIG. 5.

In an embodiment, the process 600 includes selecting 604 applicablerules based on the plurality of parameters. The selection may be basedon one, two or more of the plurality of parameters obtained above. Forexample, different sets of rules may be applicable to different sets ofparameters. As another example, the applicable set of rules may bedictated by a subset of the plurality of parameters such as whether thedevice identifier is associated with a desktop or a mobile device, theoperating system or processor architecture associated with the deviceidentifier and the like.

In an embodiment, the process 600 includes applying the selected rulesto determine 606 a plurality of weight values respectively associatedwith at least some of the plurality of parameters. For example, for eachparameter value associated with a given parameter, a value-weight mapmay be used to look up a corresponding weight value. The value-weightmap may be stored as a lookup table or similar data structure in memoryor in another data storage medium. The value-weight map(s) may or maynot be part of the rules obtained in step 604. The weight values may beused to obtain 608 a device quality score. In some embodiments, weightvalues may be further weighted before being combined to derive thedevice quality score. For example, the device quality score may includea linear combination of the weighted weight values. The formula oralgorithm for combining the weight values may or may not be part of therules obtained in step 604. The quality score may be represented bynumeric value. The numeric value may fall within a predeterminednumerical range. In general, a higher quality score indicates a higherlikelihood that a device identifier is unique and vice versa.

In some embodiments, the quality score may be used to derive 610 aquality grade, such as grade A, B, C and so on. For example, a qualityscore between 80 and 100 may be categorized as quality grade A, aquality score between 60 and 80 may be categorized as quality grade B,and so on. The quality grade may be represented by numeric ornon-numeric values. In some embodiments, the step 610 of deriving aquality grade may be optional.

FIG. 7 illustrates an example process 700 for calculating a devicequality score, in accordance with an embodiment. The process 700 may besimilar to the process 600 described above in connection with FIG. 6.

The illustrated process may be used to calculate the quality scoreassociated with a transaction or network event 702 that is identified bya device identifier (not shown). A plurality of parameters may beassociated with the network event, such as user agent identifier 704, IPaddress 706, language code 708, and the like. For each of the pluralityof parameters, a weight value may be obtained based on a correspondingvalue-weight map. For example, the value-weight maps 716, 718, 720 maycorrespond respectively to the parameters 704, 706 and 708. In someembodiments, the parameter values may be transformed or otherwise usedto obtain the parameter value used to look up the value-weight map, forexample, according to parameter-specific usage rules 710, 712 and 714.For example, the length of the character string of a user agentidentifier is to be calculated and used as the parameter value to lookup the user agent value-weight map 716, according to rule 710. Forexample, a user agent identifier “UA_2.8.1” has a character length of 8and a weight value of 0.5 according to the user agent value-weight map716. For IP addresses, no transformation may be necessary and theoriginal IP address may be used to look up the IP address value-weightmap 618, according to rule 712. For example, an IP address “310.31.345”has an exact match in the IP address value-weight map 718 with a weightvalue of 0.4. And for language codes, the number of word count may beused to look up the language value-weight map 720, according to rule714. For example, a language “Fr-fr” has a word count of 2 and a weightvalue of 0.02 according to the language value-weight map 720.

Once the weight values are obtained, they may be combined to derive thedevice quality score, for example, according to a formula and/oralgorithm 722 to derive the final quality score 724. As discussed above,the weight values may be further weighted (e.g., according to therelative significance of the parameters) before they are combined. Insome embodiments, the quality score may be normalized, for example,using a coefficient. Any suitable methods of normalization may be usedto normalize the quality score.

In various embodiments, the usage rules, value-weight maps, formula oralgorithm discussed above may collectively comprise the evaluation rulesdiscussed herein. Such evaluation rules may be derived once based onhistorical data and used repeatedly to quickly calculate quality scoresfor many device identifiers. The evaluation rules may be updated, forexample, on a periodic basis, as the historical data evolves.

FIG. 8 illustrates an example process 800 for determining the uniquenessof a device identifier, in accordance with an embodiment. Aspects of theprocess 800 may be performed, for example, by the device identificationsystem discussed in connection with FIG. 2a or 2 b.

In an embodiment, process 800 includes detecting 802 a network eventsuch as a browser event (e.g., clicking of a control, scrolling,resizing, loading or closing of a web page, and the like). Suchdetection may be implemented by a browser script (e.g., JavaScript).

In an embodiment, in response to the detected network event, a pluralityof parameters related to the device associated with the network event isobtained 804. Based on some or all of the obtained plurality ofparameters, a device identifier and a device quality grade may bedetermined 806. Such determination may be further based onpre-calculated rules derived from historical data. The rules may beembodied by the value-weight map/lookup table discussed herein. Finally,the device identifier and the device quality grade may be stored 808,such as in a data store 220 discussed in connection with FIG. 2b . Insome embodiments, storage 808 step may be optional and the deviceidentifier and device quality grade may be used directly without beingstored first.

In some embodiments, the present invention separates the calculation ofrules based on historical data from the use of the rules to derivedevice identifier and/or device quality score or grade associated withthe device identifier. Such separation serves to limit the access to thehistorical data alleviating concerns with respect to the privacy of thehistorical data. The separation also allows fast, real-time or nearlyreal time, and scalable determination of device identifiers and/ordevice quality scores and/or grades. Additionally, in some embodiments,the rules may be stored at a central location, making it easy tomaintain and update the rules independently from the uses of the rules.

While preferred embodiments of the present invention have been shown anddescribed herein, it will be obvious to those skilled in the art thatsuch embodiments are provided by way of example only. Numerousvariations, changes, and substitutions will now occur to those skilledin the art without departing from the invention. It should be understoodthat various alternatives to the embodiments of the invention describedherein may be employed in practicing the invention. It is intended thatthe following claims define the scope of the invention and that methodsand structures within the scope of these claims and their equivalents becovered thereby.

What is claimed is:
 1. A computer-implemented method for automated selection of an electronic security action based on a uniqueness assessment of a device identifier, said computer-implemented method under the control of one or more computer systems configured with executable instructions and comprising: establishing an electronic connection with one or more databases storing at least: a plurality of evaluation rules; and a predetermined threshold associated with a range of quality scores, wherein the predetermined threshold is associated with a predetermined threshold value; detecting a first online session between a computer system and a first user device; detecting a network event during the first online session; receiving device data associated with the first user device, wherein the device data is captured during the first online session and comprises a plurality of parameters comprising a first parameter and a second parameter, and wherein the plurality of parameters are associated with the network event; determining a first device identifier associated with the first user device based at least in part on the plurality of parameters; receiving a first plurality of evaluation rules from the one or more databases, wherein the first plurality of evaluation rules are selected based at least in part on a value of the first parameter; determining, based at least in part on the first plurality of evaluation rules and values of the plurality of parameters, a plurality of weight values, wherein the plurality of weight values are associated with the plurality of parameters; calculating, based at least in part on the plurality of weight values, a first quality score associated with the first device identifier; receiving an electronic indication that the first online session is likely fraudulent; accessing the predetermined threshold from the one or more databases; comparing the first quality score with the predetermined threshold value; determining that the first quality score satisfies the predetermined threshold, wherein the determination that the first quality score satisfies the predetermined threshold indicates that the first device identifier is not shared by multiple devices; and based at least in part on the determination that the first quality score satisfies the predetermined threshold, adding the first device identifier to a blacklist such that network traffic from any devices with an association to the first device identifier are blocked.
 2. The computer-implemented method of claim 1, wherein the plurality of evaluation rules are generated using a machine learning technique, wherein the machine learning technique comprises a neutral network or a statistical analysis of historical data comprising device fingerprints and indication of whether such fingerprints uniquely identify a device.
 3. The computer-implemented method of claim 2, wherein the calculating the first quality score is performed without access to the historical data.
 4. The computer-implemented method of claim 2, wherein the plurality of evaluation rules is updated periodically based at least in part on changes of the historical data.
 5. The computer-implemented method of claim 1, wherein: the plurality of weight values comprises: a first weight value associated with the first parameter; and a second weight value associated with the second parameter; the first weight value is determined based at least in part on a first weight-value map and the value of the first parameter; and the second weight value is determined based at least in part on a second weight-value map and a value of the second parameter.
 6. The computer-implemented method of claim 5, wherein: the plurality of evaluation rules comprises a first evaluation rule and a second evaluation rule; the first evaluation rule is associated with the first parameter and the first weight-value map; and the second evaluation rule is associated with the second parameter and the second weight-value map.
 7. The computer-implemented method of claim 1 further comprising: based at least in part on the determination that the first quality score satisfies the predetermined threshold, transmitting a targeted content to the devices with an association to the first device identifier, wherein the targeted content is customized based at least in part on the device data associated with the first user device.
 8. The computer-implemented method of claim 1 further comprising: determining that the first quality score does not satisfy the predetermined threshold, wherein the determination that the first quality score does not satisfy the predetermined threshold indicates that the first device identifier is shared by multiple devices; and based at least in part on the determination that the first quality score does not satisfy the predetermined threshold, monitoring network traffic from the devices with an association to the first device identifier.
 9. A computer system for automatically selecting an electronic security action based on a uniqueness assessment of a device identifier, the system comprising: one or more processors; and a memory, including instructions executable by the one or more processors to cause the computer system to at least: establish an electronic connection with one or more database storing at least: a plurality of evaluation rules; and a predetermined threshold associated with a range of quality scores, wherein the predetermined threshold is associated with a predetermined threshold value; detect a first online session between a computer system and a first user device; detect a network event during the first online session; receive device data associated with the first user device, wherein the device data is captured during the first online session and comprises a plurality of parameters comprising a first parameter and a second parameter, and wherein the plurality of parameters are associated with the network event; determine a first device identifier associated with the first user device based at least in part on the plurality of parameters; receive a first plurality of evaluation rules from the one or more databases, wherein the first plurality of evaluation rules are selected based at least in part on a value of the first parameter; determine, based at least in part on the first plurality of evaluation rules and values of the plurality of parameters, a plurality of weight values, wherein the plurality of weight values are associated with the plurality of parameters; calculate, based at least in part on the plurality of weight values, a first quality score associated with the first device identifier; receive an electronic indication that the first online session is likely fraudulent; access the predetermined threshold from the one or more databases; compare the first quality score with the predetermined threshold value; determine that the first quality score satisfies the predetermined threshold, wherein the determination that the first quality score satisfies the predetermined threshold indicates that the first device identifier is not shared by multiple devices; and based at least in part on the determination that the first quality score satisfies the predetermined threshold, add the first device identifier to a blacklist such that network traffic from any devices with an association to the first device identifier are blocked.
 10. The system of claim 9, wherein the plurality of evaluation rules are generated using a machine learning technique, wherein the machine learning technique comprises a neutral network or a statistical analysis of historical data comprising device fingerprints and indication of whether such fingerprints uniquely identify a device.
 11. The system of claim 10, wherein the plurality of evaluation rules is updated periodically based at least in part on changes of the historical data.
 12. The system of claim 10, wherein the first quality score is calculated without access to the historical data.
 13. The system of claim 9, wherein the instructions executable by the one or more processors further cause the computer system to: based at least in part on the determination that the first quality score satisfies the predetermined threshold, transmit a targeted content to the devices with an association to the first device identifier, wherein the targeted content is customized based at least in part on the device data associated with the first user device.
 14. The system of claim 9, the instructions executable by the one or more processors further cause the computer system to: determine that the first quality score does not satisfy the predetermined threshold, wherein the determination that the first quality score does not satisfy the predetermined threshold indicates that the first device identifier is shared by multiple devices; and based at least in part on the determination that the first quality score does not satisfy the predetermined threshold, monitor network traffic from the devices with an association to the first device identifier.
 15. A non-transitory computer storage having stored thereon a computer program, the computer program including executable instructions that instruct a computer system to at least: establish an electronic connection with one or more databases storing at least: a plurality of evaluation rules; and a predetermined threshold associated with a range of quality scores, wherein the predetermined threshold is associated with a predetermined threshold value; detect a first online session between a computer system and a first user device; detect a network event during the first online session; receive device data associated with the first user device, wherein the device data is captured during the first online session and comprises a plurality of parameters comprising a first parameter and a second parameter, and wherein the plurality of parameters are associated with the network event; determine a first device identifier associated with the first user device based at least in part on the plurality of parameters; receive a first plurality of evaluation rules from the one or more databases, wherein the first plurality of evaluation rules are selected based at least in part on a value of the first parameter; determine, based at least in part on the first plurality of evaluation rules and values of the plurality of parameters, a plurality of weight values, wherein the plurality of weight values are associated with the plurality of parameters; calculate, based at least in part on the plurality of weight values, a first quality score associated with the first device identifier; receive an electronic indication that the first online session is likely fraudulent; access the predetermined threshold from the one or more databases; compare the first quality score with the predetermined threshold value; determine that the first quality score satisfies the predetermined threshold, wherein the determination that the first quality score satisfies the predetermined threshold indicates that the first device identifier is not shared by multiple devices; and based at least in part on the determination that the first quality score satisfies the predetermined threshold, add the first device identifier to a blacklist such that network traffic from any devices with an association to the first device identifier are blocked.
 16. The non-transitory computer storage of claim 15, wherein the plurality of evaluation rules are generated using a machine learning technique, wherein the machine learning technique comprises a neutral network or a statistical analysis of historical data comprising device fingerprints and indication of whether such fingerprints uniquely identify a device.
 17. The non-transitory computer storage of claim 16, wherein the plurality of evaluation rules is updated periodically based at least in part on changes of the historical data.
 18. The non-transitory computer storage of claim 16, wherein the first quality score is calculated without access to the historical data.
 19. The non-transitory computer storage of claim 15, wherein the executable instructions further instruct the computer system to: based at least in part on the determination that the first quality score satisfies the predetermined threshold, transmit a targeted content to the devices with an association to the first device identifier, wherein the targeted content is customized based at least in part on the device data associated with the first user device.
 20. The non-transitory computer storage of claim 15, wherein the executable instructions further instruct the computer system to: determine that the first quality score does not satisfy the predetermined threshold, wherein the determination that the first quality score does not satisfy the predetermined threshold indicates that the first device identifier is shared by multiple devices; and based at least in part on the determination that the first quality score does not satisfy the predetermined threshold, monitor network traffic from the devices with an association to the first device identifier. 